Suchergebnisse

AbstractThis article surveyed different emerging technologies (ET), in particular artificial intelligence, and their burgeoning application in the evaluation industry. Evidence suggests that evaluators have been relatively slow in adopting ET in their practice. However, more recent data suggest that ET adoption is increasing. This article then analyzed if, and how, ET affect the evaluation industry and evaluation practice. The article finds that program evaluation is one of several competing forms of knowledge production informing decision‐making, particularly in the government and not‐for‐profit sectors. Therefore, evaluation faces a number of challenges stemming from ET. In this article, it is argued that evaluators must, albeit critically, embrace ET. Most likely, ET will complement evaluation practice and, in some instances, displace human tasks.

AbstractFor some time now, evaluators have been trying to locate their work in relation to the emergence of performance management. Although some have rejected performance management outright as conceptually weak and simplistic, others have looked for complementarities between the two approaches to generating knowledge. The authors address these concerns and identify the emergence of complementarity below the broad constructs of evaluation and performance management writ large, instead seeing it as inhering in the approaches to measurement and monitoring employed by practitioners of these disciplines, respectively. This chapter elucidates performance management and the six key elements it requires: leaders, managers, accountability systems, performance budgeting, measuring and monitoring, and evaluation. It also indicates some of the major concerns evaluators have raised regarding the validity of knowledge produced within performance‐management approaches that do not rely on evaluations. © Wiley Periodicals, Inc., and the American Evaluation Association.

AbstractThe present chapter provides the conceptual foundation on which the remainder of this special issue is grounded. First, the chapter considers the idea of an evaluation marketplace and reflects on the limited attention awarded to the commercial aspects of evaluation. Second, the scarce—yet significant—literature on the evaluation market and industry is considered, identifying three distinct strands of contributions: the evaluation market composition, market dynamics, and strategies for navigating these. In the third section, the chapter offers working definitions of the evaluation market and evaluation services as well as presents the evaluation market framework, an emerging framework for better understanding the central components of the evaluation marketplace, including its context, composition, and dynamics. A brief set of concluding notes brings the chapter to a close.

This article focuses on what some sceptics see as disillusionment with conventional evaluation practice, in that many governments experience only limited use of evaluation findings.This has contributed to a significant increase in results-based performance measurement. Yet not everyone in the evaluation community welcomes this development.The authors make the point that significant complementarities exist between evaluation and performance measurement and therefore the boundaries between these practices may need to be redefined. In other words, evaluators will need to enter into a constructive dialogue with performance management practitioners. By investigating their methodological similarities and differences, the authors argue that evaluation studies and performance measurement are highly complementary forms of knowledge production. Finally, they argue that evaluation tools may in fact strengthen a number of the identified shortcomings of performance measurement systems when applied in performance management.

Cover -- Title Page -- Copyright -- Contents -- List of Illustrations -- List of Tables -- Foreword -- Acknowledgements -- 1 Introduction -- Part One: Evaluative Approaches to Find Success -- 2 Positive Thinking and Learning from Evaluation -- 3 Structuring Evaluations and Monitoring for Learning -- 4 Using Big Data to Identify Success -- 5 What Public Sector Organizations Can Learn from the Private Sector Evaluative Practices: The Case of the Balanced Scorecard -- 6 Focusing on Success: A Review of Everyday Practices of Organizational Learning in Public Administration

AbstractThis chapter outlines the development of performance monitoring and program evaluation at the federal level of the government of Canada. This approach stands out, as it has had a dual emphasis on both monitoring and evaluation as complementary forms of knowledge production. However, throughout its history emphases on management and accountability have shifted. The authors argue that to fulfill its potential to support results‐based management the Canadian government must adopt a more stable, balanced, and strategic approach to both performance monitoring and program evaluation. © Wiley Periodicals, Inc., and the American Evaluation Association.

Realist evaluation is an established approach in evaluation. The main question driving realist evaluation is to uncover how, for whom, and under what conditions an intervention works. This is accomplished by empirically examining the inner mechanisms by which an intervention generates outcomes within a particular context—explicating the underlying context–mechanism–outcome configurations. Despite the central role of context in realist evaluation, systematic knowledge about how context is conceptualized and applied in realist evaluation is limited. Informed by the findings of a comprehensive review, the aim of this article is to unpack how context is conceptualized in realist evaluations, the types of contextual factors examined as part of context–mechanism–outcome configurations, and the methodological challenges experienced by evaluators when examining these in realist evaluations. The article concludes with a discussion of conceptual and practical developments for future applications of realist evaluations.

This article reports on a study that explored whether evidence can be found of a shared evaluation tradition among evaluation researchers and practitioners working in institutions in the Nordic countries. The study focused on articles in peer-reviewed, international, designated evaluation journals in the period 2000–12; it found little evidence from the analysis of these sources to support this claim. Meanwhile, the study found a clear preference of Nordic evaluators for publishing in European journals, with Sweden being the dominant source country in terms of number of publications, selection of journals in which they were published, and institutions and authors publishing the most.

AbstractThis chapter situates findings and insights from the case studies in this issue of New Directions for Evaluation in relation to ongoing debates among evaluators pertaining to performance management. It highlights ways in which the complementarity between evaluation and performance management has been described in the case studies and identifies five types of complementarity between performance measurement and evaluation: sequential, informational, organizational, methodical, and hierarchical. It concludes with a survey of the literature on the challenges in implementing performance management and argues that evaluators need to take a more active role in performance‐management efforts. © Wiley Periodicals, Inc., and the American Evaluation Association.

Realist evaluation and experimental designs are both well-established approaches to evaluation. Over the past 10 years, realist trials—evaluations purposefully combining realist evaluation and experimental designs—have emerged. Informed by a comprehensive review of published realist trials, this article examines to what extent and how realist trials align with quality standards for realist evaluations and randomized controlled trials and to what extent and how the realist and trial aspects of realist trials are integrated. We identified only few examples that met high-quality standards for both experimental and realist studies and that merged the two designs.

AbstractThis chapter reflects on the findings from a comparative analysis of the cases comprising this issue, summarizing and expanding on key findings from these in relation to the evaluation market framework. Major implications of the findings for evaluation practice are considered. The chapter concludes with an outline of a future research agenda for the evaluation marketplace.

AbstractThis chapter investigates the case of the Center for Employment Opportunities (CEO), a not‐for‐profit organization dedicated to providing immediate and comprehensive employment services to persons with criminal convictions. CEO prepares returning prisoners for the workforce with strong employment‐retention outcomes and proven impacts (in a randomized control trial by MDRC) showing significant reduction in recidivism. In this chapter the authors survey how CEO manages its performance through the use of both performance measurement and monitoring and evaluation. Also, the authors argue that performance measurement and evaluation must be seen as complementary forms of knowledge production and that both need to be integral to performance management. © Wiley Periodicals, Inc., and the American Evaluation Association.